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What is claimed is: 

1. A surveillance system comprising: 
a camera arranged to output images of a 

protected area; 

an input device arranged to provide a data 
annotation; and, 

a server arranged to synchronously store the 
images and the data annotation so that the data 
annotation can be used to search for a segment of the 
images . 

2. The surveillance system of claim 1 wherein 
the server is arranged to time stamp the data annotation. 

15 

3. The surveillance system of claim 2 wherein 
the server is arranged to compare the time stamp of the 
data annotation to an image count when searching for the 
segment of the images. 

20 

4. The surveillance system of claim 3 wherein 
the server is arranged to cause the segment of the images 
matching the time stamp to be displayed. 
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5. The surveillance system of claim 1 wherein 
the server is arranged to save the data annotation in SQL 
readable form. 

5 6. The surveillance system of claim 5 wherein 

the server is arranged to time stamp the data annotation. 

7 . The surveillance system of claim 5 wherein 
the server is arranged to receive an SQL search string 

10 corresponding to the data annotation to be searched and 

to search for the data annotation based on the SQL search 
string . 

8. The surveillance system of claim 7 wherein 
15 the server is arranged to match the data annotation found 

as a result of the search to the segment of the images. 

9. The surveillance system of claim 6 wherein 
the server is arranged to match the time stamp of the 

20 data annotation to an image count when searching for the 
segment of the images. 
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10. The surveillance system of claim 1 wherein 
the camera comprises a video camera, and wherein the 
server comprises a video server. 

5 11. The surveillance system of claim 1 wherein 

the camera comprises an IR camera. 

12. The surveillance system of claim 1 wherein 
the camera comprises a thermal imager. 

10 

13. A method comprising: 

storing surveillance video in a memory; 

storing data annotations in the memory, wherein 
the data annotations are useful in searching for a video 
15 segment of the surveillance video of interest; and, 

synchronizing the stored data annotations to 
the corresponding video segments of the stored video so 
that the data annotations can be used to search for the 
video segment of interest. 

20 

14. The method of claim 13 wherein the 
synchronizing of the stored data annotations to the 
corresponding video segments comprises time stamping the 
data annotations with corresponding time stamps. 
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15. The method of claim 14 further comprising 
searching for a particular data annotation. 

5 16. The method of claim 15 further comprising 

comparing the time stamp of the particular data 
annotation to a timing of the video when searching for 
the video segment of interest. 

10 17. The method of claim 16 further comprising 

displaying the video segment of interest that matches the 
time stamp of the particular data annotation. 

18. The method of claim 13 further comprising 
15 searching the data annotations using a search criteria. 

19. The method of claim 18 wherein the 
synchronizing of the stored data annotations to the 
corresponding video segments comprises time stamping the 

20 data annotations with corresponding time stamps. 

20. The method of claim 19 further comprising 
searching for a particular data annotation. 
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21. The method of claim 20 wherein the 
searching includes matching the time stamp of the 
particular data annotation to a timing of the video. 

22. The method of claim 21 further comprising 
displaying the video segment of interest that matches the 
time stamp of the particular data annotation. 

23. The method of claim 18 wherein the 
searching of the data comprises using SQL to conduct the 
search . 

24. The method of claim 23 wherein the 
synchronizing of the stored data annotations to the 
corresponding video segments of the stored video 
comprises time stamping the data annotations with 
corresponding time stamps, wherein the searching 
comprises matching the time stamp associated with the 
stored data annotation that corresponds to the search 
criteria to a timing of the video, and wherein the method 
further comprises displaying the video segment of 
interest whose timing matches the time stamp associated 
with the stored data annotation that corresponds to the 
search criteria. 
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25. A surveillance method comprising: 
capturing images of a protected area; 
storing the images in a computer readable 

memory; 

storing data annotations in the computer 
readable memory, wherein the data annotations are 
searchable using a search criteria; and, 

storing a link that links the stored data 
annotations to corresponding image segments of the stored 
images so that the data annotations can be used to search 
for an image segment of interest. 

26. The surveillance method of claim 25 
further comprising searching for a particular data 
annotation using the search criteria. 

27. The surveillance method of claim 26 
further comprising displaying the image segment of 
interest linked to the particular data annotation found 
as a result of the search. 

28. The surveillance method of claim 25 
wherein the link comprises a time stamp. 
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29. The surveillance method of claim 28 
further comprising searching for a particular data 
annotation using the search criteria. 

5 

30. The surveillance method of claim 29 
further comprising comparing the time stamp corresponding 
to the particular data annotation found as a result of 
the search to a timing of the images when searching for 

10 the image segment of interest. 



31. The surveillance method of claim 30 
further comprising displaying the image segment of 
interest whose timing matches the time stamp of the 
15 particular data annotation. 



32. The surveillance method of claim 25 
wherein the search criteria comprises an SQL search 
criteria . 

20 
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33. The surveillance method of claim 32 
further comprising : 

searching for a particular data annotation 
using the SQL search criteria; 

finding the image segment of interest linked to 
the particular data annotation; and, 

displaying the image segment of interest. 

34. The surveillance method of claim 25 
wherein the link comprises a data attribute stamp, and 
wherein the data attribute serves as an index to retrieve 
video and data segments of the same characteristic 
inferred by the data attribute. 

35. The surveillance method of claim 34 
wherein the data attribute comprises temperature. 

36. The surveillance method of claim 34 
wherein the data attribute comprises luminosity. 

37. The surveillance method of claim 34 
wherein the data attribute comprises a biometric 
signature . 
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